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3-D  Displays  and 
Controls  for  Sonar 
Operators 


Advances  in  human- 
computer  interface 
technology  enable 
improved  display  and 
computer  control. 
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DAVID  ROUSSEAU 

Advanced  Technology 
ASW  Displays  Project 
Nary  Command,  Control,  and  Ocean 
Surveillance  Center 
Research,  Development,  Test  and 
Evaluation  Division 
S an  Diego,  Calif. 


The  next  generation  of  human- 
computer  interface  technology  is 
rapidly  evolving  to  permit  a  new  way 
of  displaying  information  and  con¬ 
trolling  computers. 

•  The  increasing  speed  and 
decreasing  cost  of  high-resolution 
graphics  computers  is  making  it  prac¬ 
tical  to  generate  complex  imagery 
with  acceptable  detail  in  real  time. 

•  Small  high-resolution  mono¬ 
chrome  cathode  ray  tubes  (CRTs) 
have  long  been  used  for  military  hel¬ 
met-mounted  displays  (HMDs). 
These  are  presently  among  the  high¬ 


est  resolution  displays  used  in  HMDs. 
Full  color  CRT-based  systems  are  just 
now  becoming  commercially  avail¬ 
able,  and  color  liquid-crystal  displays 
(LCDs)  with  acceptable  resolution 
should  be  available  by  mid-decade. 
Both  types  of  displays  are  already 
finding  their  way  into  commercial 
stereoscopic  HMD  systems. 

•  Commercial  versions  of  electro¬ 
magnetic  and  infrared  (IR)  six- 
degree-of-freedom  (6-DOF)  position 
sensors  were  originally  developed  for 
military  head-trackers  that  control 
aircraft  sensors  and  weapons.  These, 
and  ultrasonic,  systems  are  now 
becoming  increasingly  affordable. 
Their  use  has  also  been  expanded  to 
create  various  three-dimensional  (3- 
D)  haptic  controllers. 

•  Automatic  speech  recognition 
systems  have  been  under  develop¬ 
ment  for  years  and  are  finally  becom¬ 
ing  fast  and  robust  enough  for  serious 
applications.  Speech  recognition  sys¬ 
tems  have  evolved  into  continuous- 
speech  and  discrete-word  versions. 


The  continuous  speech  systems  must 
be  extensively  "trained"  by  each  user 
in  order  to  successfully  function.  The 
discrete-word  versions  require  much 
less  training  and  can  accommodate  a 
wider  spectrum  of  individuals,  but 
the  speaker  must  deliberately  insert 
short  pauses  between  each  word  spo¬ 
ken.  There  is  one  exception  to  this 
dichotomy,  however.  For  the  last  few 
years,  BB&N  has  been  working  under 
Defense  Advanced  Research  Projects 
Agency  (DARPA — recently  renamed 
ARPA  to  increase  its  non-defense 
technology  development  efforts) 
funding  to  develop  a  speaker  inde¬ 
pendent  continuous-speech  recogni¬ 
tion  system.  They  have  developed  a 
software  package  that  is  presently 
undergoing  beta-testing  on  Silicon 
Graphics  INDIGO  computers.  Recent 
demonstrations  have  shown  very 
good  results.  [Ed.  note:  Although 
orthogonal  in  principle,  continuous 
speech,  speaker-independence,  and 
large  vocabulary  size  all  make  the 
recognition  task  more  challenging. 
For  more  on  speech  recognition  and 
VR,  see  the  Thomas  &  Stuart  article  in 
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Virtual  Reality  Systems  Vol.  1,  No.  1.] 

•  Two  methods  of  performing 
eye-traeking  are  prevalent  and  com¬ 
mercially  available.  The  most  com¬ 
mon  method  uses  IR  light  and  1R 
video  cameras  that  monitor  the  eye. 
The  orientation,  or  "look-  vector,"  of 
the  eye  is  computed  from  the  relative 
positions  of  the  digitized  video 
imagery  of  the  pupil  and  Purkinje 
reflection.  A  newer  technique  to  reach 
the  commercial  market  uses  comput¬ 
ers  to  monitor  the  neural  impulses 
from  the  muscles  that  control  eye 
movement.  This  system,  called  "elec¬ 
tro-oculography,"  is  like  a  localized 
electroencephalogram  (EEG)  that  uses 
simple  contacts  around  the  brow, 
temple  and  cheek  to  detect  the  activi¬ 
ty  of  the  eye-control  muscles. 

Finally,  the  newest  technology  on 
the  scene  is  digitally  modified  sound 
that  presents  the  operator  with  3-D 
binaural  hearing  via  headphones. 
This  system  modulates  the  frequency, 
amplitude  and  timing  of  sound  so 
that  it  is  altered  in  the  same  way  that 
it  would  have  been  if  the  listener  had 
actually  heard  it  coming  from  the 
desired  location.  The  computer 
processes  sound  through  digital  sig¬ 
nal  processors  executing  equations 
that  are  empirically  determined  and 
which  replicate  the  acoustic  effects  of 
the  shape  of  the  outer  ear.  This  sys¬ 
tem  is  based  on  university  and  NASA 
research  which  leads  directly  to  a 
commercial  product  that  can  present 
multiple  channels  of  realistic  3-D 
audio. 

These  technologies  are  being 
studied,  individually  and  collectively, 
for  their  application  to  tactical  Naval 
operations  at  the  Navy  Command, 
Control  and  Ocean  Surveillance 
Center  (NCCOSC)  Research, 
Development,  Test  and  Evaluation 
Division  in  San  Diego.  Many  of 
today's  Virtual  Reality  applications 


don't  require  all  these  technologies  to 
be  successful,  but  some  applications 
do.  Anti-submarine  warfare  (ASW)  is 
one  application  area  that  will  require 
the  best  of  all  these. 

Discussion 

It  is  known  that  performance 
with  displays  improves  when  opera¬ 
tor  workload  is  reduced,  and  that  this 
effect  is  amplified  by  prolonged  oper¬ 
ations.  Therefore,  displays  which 
minimize  this  work  load  result  in 
reduced  errors  and  fatigue,  improved 
operator  speed,  and  improved  deci¬ 
sion  making.  This  requires  displays 
that  present  information  in  ways 
which  take  maximum  advantage  of 
natural  human  perceptual  and  cogni¬ 
tive  skills.  Few  display  systems  in  use 
today  are  more  abstract  than  those 
used  to  support  the  ASW  mission 
area. 

Sonar  displays  have  not  changed 
significantly  for  several  years. 
Existing  ASW  displays  present  infor¬ 
mation  that  is  altered,  deleted,  or  sim¬ 


plified  due  to  the  physical  limitations 
of  the  display  devices  themselves.  All 
sonar  information  is  presented  on 
monochrome  computer  screens  con¬ 
trolled  by  a  host  of  variable  function 
keys.  The  sonar  data  presented  on 
any  given  display  is  chosen  from  one 
horizontal  "Slice"  of  the  sensor  data. 
Passive  sonar  data  is  displayed  in 
bearing  vs.  time  format,  while  active 
data  is  presented  in  bearing  vs.  range 
format.  There  is  no  map  of  the  ocean 
on  the  screen  for  reference,  and  the 
sonar  operator  can  only  listen  to  one 
beam  of  the  sonar  at  a  time.  Watch 
Jonesy  do  his  job  in  The  Hunt  for  Red 
October  to  get  a  glimpse  of  what  it's 
like.  Diverse  sources  of  information 
must  be  correlated  by  the  sonar  oper¬ 
ator,  who  then  creates  a  mental  model 
of  the  multi-dimensional  ASW  envi¬ 
ronment.  The  majority  of  this  mental 
image  must  be  derived  from  multi¬ 
dimensional  acoustic,  physical,  and 
temporal  information  that  has  been 
presented  in  a  very  abstract  visual 
two-dimensional  (2-D)  format  (Figure 


Figure  1.  Today's  method — Multidimensional  audio 
information  is  presented  as  abstract  visual  data  in  2-D. 
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Figura  2.  Rail-time  Active  Propagation  Display. 


for  presentation  of  multiple  beams 
from  the  sonar  to  provide  intuitive 
cuing,  and  for  correlation  of  acoustic 
transients  and  active  returns  to  the 
ocean  environment,  and 

5)  the  intuitive  depiction  of  high- 
resolution  computer-generated 
imagery  of  the  ASW  environment 
based  on  the  display  of  active,  pas¬ 
sive,  location,  and  environmental  sen¬ 
sor  information  integrated  with  the 
local  bottom  topography  database. 

The  exact  nature  of  the  displays 
that  may  prove  most  effective  is  yet  to 
be  determined,  but  speculation  unfet¬ 
tered  by  display  constraints  leads  to 
some  intriguing  possibilities 


1).  This  method  of  presenting  com¬ 
plex,  cross-sensory,  multi-  dimension¬ 
al  information  on  abstract  2-D  dis¬ 
plays  may  be  enhanced  with  the  data 
from  the  ship's  sensors. 

The  application  of  new  3-D  visu¬ 
al  and  audio  display  technologies  to 
the  ASW  problem  could  yield  signifi¬ 
cant  improvements  in  operator  per¬ 
formance  and  operational  effective¬ 
ness.  A  virtual  3-D  sonar  display 
system  could  present  the  critical  tacti¬ 
cal  information  in  a  far  more  intuitive 
and  integrated  way,  thereby  reducing 
the  cognitive  interpretation  burden 
and  learning  time,  while  improving 
the  tactical  success  rate  over  current 
systems. 

Such  an  advanced  technology 
ASW  display  system  could  consist  of: 

1)  the  application  of  3-D,  stereo¬ 
scopic,  high-resolution,  color  displays 
in  an  HMD  or  boom-mounted 
(periscope-like)  system, 

2)  a  3-D  position  tracker  (for  the 


HMD), 

3)  6-DOF  manipulators  integrat¬ 
ed  with  speech  recognition  and  eye 
tracking  for  computer  function  con¬ 
trol, 

4)  the  incorporation  of  3-D  audio 


For  example,  in  the  active  sonar 
mode  the  3-D  ASW  display  could 
provide  the  sonar  operator  with  a 
360-degree  field-of-regard  image  of 
the  ocean  bottom  topography  and 
water  properties  integrated  with 
wave  fronts  representing  the  propa¬ 
gation  of  the  sonar  pulses  (Figure  2). 
The  operator  would  see  these  wave 
fronts  being  reflected  off  the  local 


display  of  sound  field  Cuing  Lines  &  Labels 


Figura  3.  Possible  real-time  3-D  passive  ASW  Display. 
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Sensor  tso-thrmtofcte'  as  affected  bv  thermal  layers 


Hours  6.  Potential  submarine  approach  officer  display. 


Therefore  the  workload  for  each  per¬ 
son  will  increase,  and  the  tactical  cost 
of  losing  one  platform  out  of  a 
reduced  inventory  will  be  magnified. 
Systems  that  can  improve  operational 
performance  while  requiring  lower 
manning  and  less  training  will  be  crit¬ 
ical,  and  those  systems  that  can 
achieve  the  greatest  performance 
improvements  at  the  lowest  cost  will 
be  essential.  n 


improve.  Reductions  in  operator 
training  and  increases  in  retention 
could  also  be  significant,  and  the 
effectiveness  of  each  sonar  operator 
could  be  multiplied  as  well. 

This  technology  could  also  be 
applied  effectively  to  ASW  tactical 
support  by  providing  the  shipboard 
ASW  officer  (or  submarine  approach 
officer)  (Figure  6)  with  a  3-D  image  of 
the  ocean  environment,  his  weapons 
engagement  envelopes,  sensor  cover¬ 
age  volumes,  and  the  hostile  sub¬ 
marines  that  have  been  identified. 

These  technologies  are  already 
being  applied  to  numerous  civilian 
tasks,  including  architecture;  mechan¬ 
ical  design  in  the  automotive  and 
aerospace  industries;  pharmaceuticals 
research;  medical  imagery  for  3-D 
display  of  CAT  scan,  NMR,  and  ultra¬ 
sonic  medical  data;  education,  and 
functional  aids  for  the  handicapped. 
If  these  new  technologies  are  applied 
to  the  tactical  naval  environment, 
their  impact  on  operational  effective¬ 
ness  could  be  dramatic,  and  their 
impact  on  training  methods  and  sys¬ 
tems  may  be  equally  dramatic. 


History  has  taught  us  that  the 
requirement  to  operate  a  global  navy 
will  not  diminish  at  a  rate  commensu¬ 
rate  with  the  decrease  in  our  assets. 
The  sale  of  Chinese  and  formerly 
Soviet  submarines  to  Third  World 
countries  continues  because  of  the 
economic  and  political  needs  of  those 
nations.  Our  budgets,  ship  invento¬ 
ries,  and  manpower  levels  will  proba¬ 
bly  continue  to  decrease  over  the  next 
decade  or  beyond,  for  similar  reasons. 
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